What ’ s the Evidence on Evidence - Based Management ?

ReviewsandExclusions Our initial search generated 216 articles. One researcher and a research assistant reviewed the abstracts to exclude easily identifiable errors of three types: duplicate entries, book reviews, and articles using the term “evidence-based” but referring to evidence-based medicine or evidencebased government policy, since these areas were not relevant to our research question. The exclusions resulted in 164 remaining articles. We then conducted a hand search to identify missed articles that met our inclusion criteria. This resulted in an additional five articles, which gave us a total of 169 articles, all of which were subjected to a full review following the process described below. ArticleReviews Since we were a research team of three, each of us read and reviewed one-third of the articles as a first step. Next, we conducted a further round of reviews so that all coding decisions (as outlined below) were made with the agreement of at least two researchers. We developed an Excel worksheet that served as our review template for each of the 169 articles. On reading and analyzing the full set of articles, we excluded another 25 articles that did not address the concept of EBMgt. The final result was 144 articles for further analysis. Each of the articles was coded according to the parameters listed in Table 2. To assess the strength of evidence related to EBMgt in a systematic way across all 144 articles, we modified a rubric for evaluating the quality of medical research developed by D’Agostino and Kwan (1995). In Table 3 we show the comparison between evaluation standards used in medical 1 Numerous assessment rubrics are used to evaluate medical research; one of the most widely used was developed by the NHS R&D Centre for Evidence-Based Medicine. The rubric we used was a somewhat less complicated one, appropriate for this assessment of the management literature, Table2 Systematic ReviewTemplate Column Title (Position) Explanation Reference number (col. A) To facilitate retrieval and secondary reviews Full reference (cols. B–G) To facilitate article retrieval Country of origin (col. H) To facilitate article review and comparisons Inclusion criteria met? (Y/N) (col. I) Q: Is this article about the concept of evidence-based management? Management area (col. J) Q: What management area does this paper address? For example, decision making, human resources and staffing, leadership, information systems, structure, strategy, marketing, general EBMgt, other (specify) Article purpose/focus (col. K) Empirical research, literature review/theory overview, theoretical framework/model, opinion piece/call to action, case study Basis of opinion (col. L) Q: If an opinion piece/call to action, what is the basis of opinion? For example, anecdotes, literature review, personal experience , other (specify) Intended target audience (col. M) For example, lay (executives, middle managers, practitioners, cross-functional/interdisciplinary decision-making team), students, policy makers, researchers, other (specify) Sector/organization (col. N) Specify targeted sector and/or organization type Level of analysis (col. O) Individual, organizational, institutional, system Methodology (col. P) Quantitative (specify), qualitative (specify), mixed methods (specify), other (specify) Data source (col. Q) Administrative, survey, interview, focus group, document review, other (specify) Type of article (col. R) Promising practices, transitional, evidence-based practice, context, mechanisms, transferability, opinion Level of evidence (col. S) A levels of evidence “rubric” used by one of us in prior studies evaluating clinical research evidence (see Barnsley et al., 2005) and adapted from the original work of D’Agostino & Kwan (1995) was used here. See Table 3 for description Comments (col. T) Free-form comments regarding article in light of study objective and comparable other articles reviewed 2009 7 Reay, Berta, and Kohn